Skip to content

platform data(known platform limits): add Copilot notes (Issue #12)#13

Merged
dacharyc merged 1 commit intoagent-ecosystem:mainfrom
rhyannonjoy:issue-12-add-copilot-known-plat-limits
Apr 3, 2026
Merged

platform data(known platform limits): add Copilot notes (Issue #12)#13
dacharyc merged 1 commit intoagent-ecosystem:mainfrom
rhyannonjoy:issue-12-add-copilot-known-plat-limits

Conversation

@rhyannonjoy
Copy link
Copy Markdown
Member

  • Resolves Issue platforms data(known platform limits): add Copilot web fetch notes #12
  • Tested on default Auto setting to explore multiple model responses to identical prompts
  • Routing and toolchain nondeterministic, curl headings confirm full retrieval while fetch_webpage performs relevance-ranking summarization/non-linear reassembly that can't be suppressed by prompt
  • No truncation limit observed, if truncation occurs, it happens before content reaches the chat window
  • More info in my Agent Ecosystem Testing docs

Copilot Test Details

Unlike API-based platforms, Copilot testing uses manual chat sessions in the VS Code IDE.
The framework generates prompts, but execution requires copy-paste into the Copilot chat
window. Both tracks test 11 distinct URLs across baseline, structured content, offset,
and edge case categories. The primary finding is that Copilot autonomously selects between
two retrieval mechanisms -fetch_webpage and/or curl - with no prompt control, and the
mechanism selected determines output format more than any other variable.

Test Category Question What it tests
Baseline What does Copilot retrieve by default? How does output vary across model routing and HTML vs Markdown URLs? MongoDB docs pages at 20KB–256KB; HTML and Markdown URL variants
Structured Content How does Copilot handle tables, code blocks, nested headings, and JavaScript-rendered pages? Wikipedia, Anthropic API docs, Markdown Guide, Google Gemini docs
Offset/Pagination Does Copilot auto-chunk after apparent truncation? Can it paginate a large document? 256KB MongoDB tutorial; fragment navigation
Edge Cases How does Copilot handle redirect chains, SPAs, raw Markdown files, and JSON endpoints? 5-level redirect chain, Gemini landing page, GitHub raw .md, httpbin.org

Copy link
Copy Markdown
Member

@dacharyc dacharyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for continuing to dig into this! I found your friction logs illuminating and definitely want to talk more about your findings there.

There is so much we don't know about how these things work with content under the hood! 😰

@dacharyc dacharyc merged commit 0b46464 into agent-ecosystem:main Apr 3, 2026
1 check passed
@rhyannonjoy rhyannonjoy deleted the issue-12-add-copilot-known-plat-limits branch April 3, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants